Multidimensional expert behavior emulation system

ABSTRACT

An expert decision-making method is emulated based on a history of behaviors by experts in a variety of observed situations. The history of behaviors is built up from observations of actions taken by experts in analyzing a plurality of situations. Situation data representative of a situation to be processed is received, and situation features are extracted from the situation data. Each situation feature is associated with an expert behavior method used to process the situation. A behavior method is recognized from a pattern of situation features. Recognizing a behavior method is based on feature/method separation data in multidimensional space of features into so areas with each area associated with a method used by experts. Parameter values for parameters in the recognized behavior method are calculated based on the situation features. The calculation of parameter values is accomplished by recognizing parameter calculation rules and calculating the parameter values using the rules. A parameter calculation rule for each parameter in the behavior method is recognized from a pattern of situation features. Recognizing a parameter calculation rule is based on feature/parameter-calculation-rules separation data of multidimensional space of features into areas with each area associated with a parameter calculation rule used by experts. The recognized behavior method is executed on the situation data using the calculated parameter values to recommend a solution for the situation. The recommended solution has solution data representing a plan of action to provide the solution and remainder data representing unprocessed situation data. A test detects whether the remainder data is in a target range. If the remainder data is not in the target range, the actions to recommend a solution are repeated until the test detects the remainder data is in the target range.

RELATED APPLICATIONS

This application is related to commonly assigned, U.S. application Ser.No. ______, entitled “TRAINING A MULTI-DIMENSIONAL, EXPERTBEHAVIOR-EMULATION SYSTEM” by and concurrently filed herewith.

TECHNICAL FIELD

This invention relates to a multi-dimensional, long-term behavior,computerized decision method and system. More particularly, theinvention relates to providing computer operations to perform routinedecisions based on the historical performance of experts in the decisionprocess.

BACKGROUND OF THE INVENTION

There are many routine decision tasks where a person receives input datafrom a computer and analyzes the data to come to a decision. In somecases, these decision tasks are multi-dimensional in input and cannoteasily be grouped into a relatively small number of classes. Because thedecisions cannot be so grouped, the decision process does not lenditself to automated or expert processes provided in traditionalrecognition or classification problems.

Further for many of these decision problems, it is difficult to measurethe quality of each individual decision made by a person, but it ispossible to measure the integral quality of a plurality of decisions asa whole made by the person over a period of time. This integral qualitycan be compared against other persons making similar decisions over aperiod of time and the relative expertise of each decision maker can bemeasured.

One example of routine decision tasks based on input data as discussedabove is in retail goods allocation tasks. In such tasks a distributionor allocation expert reviews input data on a computer display screen aquantity of retail goods to be allocated in various quantities fromwarehouses to multiple retail stores selling the goods. Where there are10 to 1000+ stores in the business and the quantity of goods to beallocated to each store varies from 0 to 100+, the number of possibleallocation outcomes can easily exceed several thousand. Such a decisionproblem is so multi-dimensional it does not lend itself to automatedsolution based on recognition and classification systems.

Further, to measure the quality of an allocation by examining a specificallocation is not meaningful. For example, if a specific set of goodssuch as swimsuits is allocated to certain stores and turns out not to beprofitable for those stores, this result may be due to weatherconditions rather than lack of experience by the allocator. On the otherhand, if over an entire season all the goods allocated by this sameallocator generate the highest total profit or other metric, this sameperson might be recognized as an expert allocator. In other words, thereis no absolutely right or wrong decision for each decision problem, butthere are the best (expert) and the poorest decision makers.

Another problem in computerizing routine decision tasks of the abovetype is that best practices in the environment of the decision problemmay change over time. For example, in the allocation of retail goods,business practices may change over time because of changes to thecompetitive environment or changes in the goals of the business entity.

SUMMARY OF THE INVENTION

In accordance with this invention an expert decision-making method isemulated based on a history of behaviors by experts in a variety ofobserved situations. A history of behaviors is built up fromobservations of actions taken by experts in analyzing a plurality ofsituations. The observations are captured, and behaviors from theobservations are constructed. The behaviors indicate an associationbetween situation features and methods with parameters for solving thesituations.

Situation data representative of a situation to be processed isreceived, and situation features are extracted from the situation data.A behavior method is recognized from a pattern of situation features.Parameter values for parameters in the recognized behavior method arecalculated based on the situation features. The recognized behaviormethod is executed on the situation data using the parameter values torecommend a solution for the situation.

The recommended solution has solution data representing a plan of actionto provide the solution and remainder data representing unprocessedsituation data. A test detects whether the remainder data is in a targetrange. If the remainder data is not in the target range, anotherbehavior method from a pattern of situation features from the remainderdata is recognized. Parameter values, based on the situation featuresfrom the remainder data, are calculated for parameters in this secondrecognized behavior method. This second recognized behavior method isexecuted on the remainder data using its parameter values to recommend asolution for the situation. These actions are repeated until the testdetects the remainder data is in the target range.

A solution strategy is displayed to a user. The strategy displayincludes all the recognized behavior methods with their parameter valuesand the solution data and remainder data for each recognized method.Adjustments to the strategy may be received from the user. Theadjustments will be changes of behavior methods or changes of parametervalues to arrive at the solution.

The above calculation of parameter values is accomplished by recognizingparameter calculation rules and calculating the parameter values usingthe rules. A parameter calculation rule for each parameter in thebehavior method is recognized from situation features. Recognizing aparameter calculation rule is based onfeature/parameter-calculation-rules separation data in a pattern offeatures in multidimensional space with each feature associated with aparameter calculation rule used by experts. Similarly, recognizing abehavior method is based on feature/method separation data in a patternof features in multidimensional space with each feature associated witha method used by experts.

The invention may be implemented as a computer process, a computingsystem or as an article of manufacture such as a computer programproduct or computer readable media. The computer program product may bea computer storage media readable by a computer system and encoding acomputer program of instructions for executing a computer process. Thecomputer program product may also be a propagated signal on a carrierreadable by a computing system and encoding a computer program ofinstructions for executing a computer process.

These and various other features as well as advantages, whichcharacterize the present invention, will be apparent from a reading ofthe following detailed description and a review of the associateddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the flow of system work in one embodiment of the invention.

FIG. 2 illustrates the computer and communication environment in whichthe present embodiments of the invention will typically operate.

FIG. 3 shows the operational flow of the usage cycle illustrated in FIG.1.

FIG. 4 shows the operational flow of the behavior emulator module 308 inFIG. 3.

FIG. 5 shows the operational flow of the recognize method module 404 inFIG. 4.

FIG. 6 shows the operational flow of the recognize parameters module 406in FIG. 4.

FIG. 7 shows the operational flow of the execute method module 408 inFIG. 4.

FIG. 8 shows the operational flow of the training cycle illustrated inFIG. 1.

FIG. 9 shows the operational flow of the behavior construction module810 in FIG. 8.

FIG. 10 shows the operational flow of the strategy extraction module 904in FIG. 9

FIG. 11 shows the operational flow of the learning module 814 in FIG. 8.

DETAILED DESCRIPTION OF THE INVENTION

In the present invention, expertise is derived from past experience ofexperts making decisions on the type of situation data being processed.The decision-making process itself is empirical as the experts havelearned what method choices applied to the situation data representing asituation provide a best plan of action to solve the situation Thechoice of method to solve the situation may or may not include a choiceof parameters to be used with that method. The expert in working withsituation data, that has multiple dimensions as discussed herein, hasbecome an expert heuristically, i.e. by trial and error, and developedan extensive personal knowledge base and historical background toanalyze and solve problems provided by the situation data.

For example, in merchandise allocation an allocator may spend six monthsor a year learning how to effectively allocate retail goods to a chainof stores. As the merchandise allocator becomes more expert, theprofitable allocation of retail goods improves for the stores. Further,some merchandise allocators seem to have more talent for doing theallocations than other allocators. Accordingly, an expert emulatingcomputer system emulating the behavior of the most talented allocatorscan be extremely valuable to a large retail company. Such a computingsystem provides expert guidance to new merchandise allocators who arenot yet expert, or provides expert guidance to lower performingallocators.

FIG. 1 shows the flow of system work in one embodiment of the invention;the user interactive work is performed at a workstation 100. Typicallythere are many workstations and a workstation may be referred to hereinas a user workstation or an expert workstation. There are two cycles ofoperation for the expert emulating system—the usage cycle 106 and thetraining cycle 108. The designation expert workstation refers to usersthat have enough expertise that observations of their actions are usedin the training cycle. A usage cycle is performed at all userworkstations, but training cycle is performed for expert workstationsonly. One or more supplier servers 102 work with the expert workstationto perform the training cycle 108, and a suppler server 103 and acustomer server 104 work with the user workstation to perform the usagecycle. “Supplier” as used herein refers to the provider of the expertbehavior emulating system. The usage cycle runs on-demand as a user atthe workstation 100 processes situation data requiring a solution. Inthe training cycle 108 retrieve observations operation 116 runsconstantly to capture observation data over a reasonable period of time.The complete loop of the training cycle runs periodically as thesupplier server 102 creates and/or updates profiles for use during theusage cycle.

In the example of a merchandise allocation system the usage cycle runson demand when an allocator requests allocation of situation data, suchas goods to be allocated to a set of stores in a chain of retailoutlets. The complete loop of the training cycle in this example mightrun only once a week or even once a year or more. The profile whichcontains allocation methods and other allocation data is normallyadjusted for long term changes in the allocation process. When a profileis first created by the training cycle, it can be expected thatadjustments and new allocation observations will be frequent and thetraining cycle will run frequently, i.e. daily or even hourly. After theprofile has been updated for several weeks, the adjustments to asolution and new observation information regarding behavior of an expertwill be minimal. The profile has settled down, and the updates to theprofiles by the training cycle will be come weekly, monthly or evenquarterly on the calendar

The usage cycle begins with retrieval operation 110 which retrieves thesituation data selected by the user and provides the situation data tothe expert behavior emulator 112. The expert emulator 112 analyzes thesituation data and makes a solution recommendation to the user. The userreviews the solution recommendation, and adjust/save operation 114 mayor may not adjust the solution based on input from the user. After afinal solution is reached and approved by the user, the solution data issaved by adjust/save operation 114 to the customer database as asolution for the situation data being analyzed. The usage cycle isdescribed in more detail hereinafter with reference to FIG. 3.

The training cycle begins with retrieval operation 116 which retrievesobservations regarding actions taken by the expert when dealing withparticular situation data. This situation data and actions informationare used by the behavior construct operation 118 to construct expertbehaviors. From these expert behaviors, profile create operation 120creates the profiles for use by the expert behavior emulator 112 in theusage cycle. The training cycle is described in more detail hereinafterwith reference to FIG. 8.

Alternatively, retrieval operation 116 and behavior construct operation118 might be performed for all users, and decisions about which usersare experts might be made later at profile create operation 120. Thisallows for better flexibility in case of a need to change a userassignment to expert or non-expert. Because all behaviors are collectedin this alternative embodiment and it is not necessary to start retrieveoperation 116 and construct behaviors operation 118 from the beginning.

FIG. 2 is illustrative of the computer and communication environment inwhich the present embodiments of the invention will typically operate.The supplier of the expert emulating system is usually located at asupplier site 200 remote from the customer site 202. The supplier siteincludes a static data server 204 and a supplier workstation 206. Thetraining cycle of FIG. 1 is performed on the supplier workstation 204and static data server 206 in the supplier site 200. At the customersite 202 there is also a supplier dynamic data server 208 that receivesthe profiles and any other administrative updates from the static dataserver 204 at the supplier site. Communication is usually over theInternet 209 as depicted in FIG. 2, but could occur over any type oftelecommunication link.

The customer site 202 also has the customer's server 210, whichmaintains the customer's database of situation data. A customer network209 at the customer site interconnects the customer's server withsupplier's dynamic data server 208 and the customer's workstations 212.The supplier also has an administrative laptop 214 connected to thecustomer network. Thus a user works at the workstations 212, and theuser's work is assisted in the methodology of recommending situationsolutions in accordance with the usage cycle discussed above for FIG. 1.Note that the supplier workstation 206 and the supplier administrativelaptop 214 are indicated as also connected to the Internet and, thus,could be remote from the supplier's static data server and thesupplier's dynamic data server 208.A computing device, such as servers,desktop computers, laptop computers, and computer workstations in FIG.2, typically includes at least some form of computer-readable media.Computer readable media can be any available media that can be accessedby the computing devices. By way of example, and not limitation,computer-readable media might comprise computer storage media andcommunication media.

Computer storage media includes volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules or other data. Computer storage media includes, but isnot limited to, RAM, ROM, EPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium that can be used tostore the desired information and that can be accessed by the computingdevices in FIG. 2.

Communication media typically embodies computer-readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media. Combinations of any ofthe above should also be included within the scope of computer-readablemedia. Computer-readable media may also be referred to as computerprogram product.

The logical operations of the various embodiments of the presentinvention are implemented (1) as a sequence of computer implemented actsor program modules running on a computing device and/or (2) asinterconnected machine logic circuits or circuit modules within thecomputing device. The implementation is a matter of choice dependent onthe performance requirements of the system implementing the invention.Accordingly, the logical operations making up the embodiments of thepresent invention described herein may be referred to variously asoperations, structural devices, acts or modules. It will be recognizedby one skilled in the art that these operations, structural devices,acts and modules may be implemented in software, in firmware, in specialpurpose digital circuit logic, and any combination thereof withoutdeviating from the spirit and scope of the present invention as recitedwithin the claims attached hereto.

The drawings in FIGS. 3-11 shall follow the illustrating convention ofoperations or modules being indicated by rectangles, decision operationsbeing indicated by diamonds, and information being indicated bytrapezoids. Further, the same item referenced in multiple drawings willbe given the same reference numeral.

FIG. 3 shows the operational flow of the usage cycle in FIG. 1 in moredetail. Receive operation 302 receives from the data 304 in thecustomers system the situation data 306 selected by a user at theworkstation. In the merchandise allocation example the situation datawould be a quantity of goods to be allocated to a plurality of storesoperated by the customer, budget and other planning information, paststore performance information, current store performance information ifavailable, any other data that might be useful for allocation decision,and the expert or user would be the merchandise allocator. The situationdata 306 is received from the customers system and stored in workingstorage at the workstation.

The behavior emulator module 308 than works with the situation data 306,the profile 310, and the subject knowledge information 312 to recommenda situation solution to the user at the workstation. The profile is ageneralization of observed decisions made by experts, where a singledecision is a sequence of methods with or without parameter values thathave in the past been used by experts as solutions in observedsituations. The subject knowledge includes a methods library, aparameters library and a features library. The methods library includesset of expert behavior methods used by the experts in the past. A methodincludes a method name, a list of required input parameters and a methodexecution code, i.e. the method operations. The parameters libraryincludes the parameter lists and limitations for the parameters in themethods; it also includes parameter calculation rules. The featureslibrary includes the features of the situation data and featurecalculation rules. The solution recommended by the behavior emulatorwill be solution data indicating a plan of action and remainder dataindicative of situation data remaining to be processed. The sequence ofmethods producing the recommended solution along with parameter valuesfor the methods is referred to herein as a strategy.

The recommended solution from and the strategy used by the behavioremulator are both displayed by display operation 316 to the user at theworkstation computer screen. A typical display would identify eachmethod used along with its parameter values, the situation solution dataand remainder data at each stage or iteration of the strategy, i.e. astrategy iteration refers to the execution of one of the methods withits parameters in a sequence of methods making up a strategy. The nextiteration works on the remainder data with the next method in thesequence. The strategy is completely executed when the remainder datarepresents a value within a predetermined range deemed acceptable for asolution to the situation.

For example in the case of merchandise allocation, the display of eachiteration of the strategy would identify a method used along with theparameter values used, the solution data, i.e. the number of goodsallocated to each store, and the remainder data, i.e. the number ofgoods unallocated. In merchandise allocation, the strategy will not becomplete usually until the remainder is zero.

When the recommended solution is displayed, the user is given anopportunity to adjust the strategy or accept the strategy. If thestrategy is accepted, accept test detects the indication from the user,and the operational flow branches to transfer operation 320. Transferoperation 320 transfers the recommended solution into the customer'sdatabase. Normally, transfer operation 320 does not transfer an entirestrategy to the customer system, because a strategy includes behaviorsteps that lead to the solution. Alternatively, the transfer operationcould transfer the strategy as well as the solution. The transferoperation always transfers the recommended solution. In the case ofmerchandise allocation, the allocation for goods, i.e. the solution,would be transferred into the customers' database and the goods would bedistributed accordingly.

If the user decides to adjust the strategy, the accept test detects thatadjustments are made and the operation flow passes to receiveadjustments operation 322. Adjust strategy operation receives theadjustment input from the user and uses that input to modify thestrategy being used by the behavior emulator. The receive operation 322also generates a corrections log 324 to record the strategy adjustmentfor subsequent use in a training cycle. After the adjust strategyoperation, the operation flow returns to the behavior emulator tore-execute the strategy which has now been modified. The modifiedstrategy 314 is used by the behavior emulator module 308 along with thesubject knowledge and the profile to recommend a new situation solution.Depending on input from the user, the behavior emulator module 308 mayjust change one method in the strategy and execute the other methods inthe strategy unchanged. Additionally, the behavior emulator may beinstructed by the user that it is allowed to adopt other methods in thestrategy subsequent to the method changed by the user.

FIG. 4 illustrates the operational flow of the behavior emulator module308 in FIG. 3. Feature extract operation 402 gets feature name andfeature calculation rules information from the subject knowledgeinformation 312 and extracts situation features from the situation data306 based on the feature calculation rules. The feature calculationrules may operate on all features or may operate on a subset of featuresto extract situation features that will be useful in recognizing themethod to be executed to provide a solution. H is also true that ifthere are no feature calculation rules for a given feature, than thatfeature in the situation data is not processed and is not used by thefeature extraction operation 402 to produce a situation feature.

Recognize method module 404 receives the situation features from featureextract operation 402, and it receives the method choices from thesubject knowledge 312 and features/methods separation data from theprofile 310. Using pattern recognition techniques in multi-dimensionalspace for all the situation features, and using the features/methodsseparation data, the recognize method module selects a method to beexecuted against the situation data.

The operation flow for the recognize method module 404 is shown in FIG.5. Get operation 502 in FIG. 5 retrieves the situation features, themethod choices and the features/methods separation data. The featuresand methods are analyzed as patterns of numeric data. Alternatively,they might be analyzed as patterns of graphical data. In any case it iseasier to understand the analysis if the situation features and thefeatures/methods separation data are imagined as a graph inmulti-dimensional space with a graph axis for the various features andgraph points for the feature values with each point labeled with amethod name. Features/methods separation data is multi-dimensionalsurface(s), which separates the entire graphical space into areaspopulated mainly by graph points labeled with one method name or nograph point at all. Each area is labeled with a method name,representing the method that prevails in that area. The methodrepresented by that area is the best choice for discriminating betweensituation features in that area, and the best method to be used to finda situation solution. Detect operation 504 will use feature values toplot a new graph point in the same multi-dimensional space. Operation504 detects the area where the new graph point appears usingfeatures/methods separation data, and select a method name associatedwith that area as the best choice for a method to operate on thesituation data.

In the example of merchandise allocation, the selected method might bean initial allocation method, and the initial quantity of retail goodsand retail stores would be the situation s features. Save methodoperation 506 retrieves the operations for the selected method and savesthe method operations 405 in working storage for subsequent use byexecute method module 408 in FIG. 4.

Recognize parameters module 406 in FIG. 4 receives the situationfeatures from the extract features operation 402, it receives theparameter lists and the limitations associated with the parameters inthe selected method from the subject knowledge 312, and it receives thefeatures/parameters-calculation-rules separation data from the profile310. The operation flow for the recognize parameters module 406 is shownin FIG. 6.

Get operation 602 in FIG. 6 retrieves the situation features, theparameter limitations, parameters calculation rules from the parameterlibrary from subject knowledge information 312. Thefeatures/parameters-calculation-rules separation data is retrieved fromthe profile 310, and the selected method is retrieved from the recognizemethod module 404. Parameter values calculation is a hierarchicalprocess. Detect operation 604 detects a parameters calculation ruleusing a technique as described at FIG. 5 for detect operation 504.Detect operation 604 uses situation features, available parametercalculation rules and features/parameter calculation rules separationdata to detect the parameters calculation rules to be used. The featuresand parameters calculation rule are analyzed as patterns of numericdata. They may also be analyzed as patterns of graphical data. Theanalysis is most easily understood if the data is imagined as beinggraphed in multi-dimensional space with a graph axis for the variousfeatures and graph points for the feature values with each point labeledwith a parameters calculation rule.Features/parameters-calculation-rules separation data ismulti-dimensional surface(s) which separates the entire graphical spaceinto areas populated mainly by graph points labeled with one rule or nograph point at all. Each area is labeled with a parameters calculationrule, representing the rule that prevails in that area. The rulerepresented by that area is the best choice for calculating parametervalues for situation features in that area. Detect operation 604 willuse feature values to plot a new graph point in the samemulti-dimensional space. Operation 604 detects the area where the newgraph point appears using features/rules separation data, and selects arule associated with that area as the best choice for calculatingparameter values for the method to operate on the situation data.

Save operation 606 saves the detected parameters calculation rule. Arules algorithm might require more parameters calculation rules for itsexecution, but some rules algorithms do not require any more parameterscalculation rules and can be executed without any additional rules. Iflast saved rule requires parameters calculation rules, then additionalrules test operation 608 branches the flow to detect operation 604, andoperations 604 and 606 repeat to detect parameters calculation rule forthis rule algorithm. If last saved rule does not require additionalparameters calculation rules, the operation flow branches to the executeoperation 610. The execute operation executes all saved parameterscalculation rules. Save operation 612 saves parameter values obtained onthe previous step, and the operation flow returns to execute methodmodule 408 in FIG. 4.

Execute method module 408 in FIG. 4 receives the situation data 306, themethod operations 405 for the selected method, and the parameter values407 for parameters in the selected method. The execute method module 408executes the method operations using the parameter values and produces asolution for the situation data. The operation flow for the executemethod module 408 is shown in FIG. 7.

The method retrieve operation 702 in FIG. 7 retrieves the methodoperations from subject knowledge 312 for the selected method fromworking storage. Parameter retrieve operation 704 retrieves theparameter values 407 for the parameters in the selected method fromworking storage. Retrieve operation 706 retrieves the situation data towhich the method and its parameter values are to be applied. Executeoperation 708 now uses the parameter values for the selected method andexecutes the method operations of the selected method on the situationdata. The result will be solution data and remainder data.

In the merchandise allocation example the solution data will be thequantity and type of goods allocated to each retail store, and theremainder data will be the remaining quantity and type of goods notallocated by the selected method. Update operation 710 saves the currentmethod, its parameters, the solution data and the remainder data, andthe operation flow returns to remainder test operation 410 in FIG. 4.

In FIG. 4 remainder test operation 410 tests whether the remainder datais in an acceptable target value range. The remainder data “R” being inan acceptable target value range might be satisfied if R is greaterthan, equal to or less than a number, or it might be satisfied if R iswithin a range of numbers or a set of ranges. Further, the number(s)might be positive or negative. If the remainder is in acceptable targetvalue range, the operation flow returns to display operation 316 in FIG.3. Display operation 316 displays the information saved by the updateoperation 710 in FIG. 7 to the user at the computer display of theworkstation.

If the remainder is not in an acceptable target range, the operationflow returns to recognize method module 404. Recognize method module404, recognize parameters module 406 and execute method module 408operate as described above except that now the situation features arelimited to those represented by the remainder data. The operational flowstays in an operational loop through modules 404, 406, 408 and testoperation 410 until R is in an acceptable target range. This completesthe description of the operation flow in the usage cycle shown in FIG.1.

The training cycle of FIG. 1 is shown in the detail training operationflow of FIG. 8. The operation flow in FIG. 8 begins with captureoperation 802 that captures observations of actions by the expert userand associates those actions with situations documented in the situationdata. The observations data 804 thus contains situations and actions.Actions refers to keyboard and mouse or other input by the expert useras she works at the workstation 100 on situation data as well asobserved data and data available from observation data screens and fromthe corrections log 324. The situation data represents a given situationfor which a solution is being sought by the expert as well as allsupportive information available to the expert for the decision-making.In the merchandise allocation example a situation would be described bythe situation data specifying the quantity of retail goods to beallocated to a set of stores in a retail chain, budget and otherplanning information, past store performance information, current storeperformance information if available, any other data might be useful forallocation decision.

Load operation 806 receives the observations 804 and extracts uniquemethods used, parameter lists and limitations, and features from thoseobservations and loads the extracted information into the subjectknowledge information 312. Load operation could be manual at thebeginning and made semi-automated as methods, parameters and featuresbecome familiar. The subject knowledge information 312 as described inthe usage operational flow includes the methods library, the parameterslibrary, and the features library. These libraries are used by thebehavior construction module 810 to construct behavior information 812.Behavior information includes features and strategies as described abovein reference to the usage cycle. The behavior construction module 812 isdescribed in more detail hereinafter with reference to FIG. 9.

After the behavior information has been constructed it is provided tothe learning module 814. The behaviors include situation features andstrategies. The learning module works with the features and strategiesto create the profile 310 of expert decisions for use by the behavioremulator. After the learning module creates the profile 310, thetraining cycle is complete. The operation of the learning module isdescribed hereinafter with reference to FIG. 11.

FIG. 9 illustrates the operational flow of the behavior constructionmodule 810 in FIG. 8. The behavior construction module performs twoimportant operations when working with the observations 804 and thesubject knowledge information 312. One operation is a feature extractionoperation 902 and the other operation is strategy extraction operation904. The subject knowledge information includes a methods library, aparameters library and a features library. The methods library includesset of methods used by experts to find solutions to prior situations. Amethod includes a method name, a list of required input parameters andmethod execution code. The parameters library includes the parameterlists and limitations for the parameters in the methods. Also itincludes parameter-calculation-rules. The features library includes thefeatures of the situation data and feature calculation rules.

Feature extraction operation 902 receives the situation data fromobservations 804 and the feature names and calculation rules from thefeatures library in the subject knowledge 312. The feature extractoperation 902 uses the feature names and feature calculation rules toextract the situation features from the situation data.

The strategy extraction module 904 receives the actions data from theobservations information 804 and it receives the method choices from themethods library and the parameter limitations for parameters in themethods from the parameters library in the subject knowledge 312. Fromthis information the strategy extraction operation determines a strategyassociated with an action. The strategy is defined by methods and theirparameter values. The strategy extraction module 904 is described inmore detail hereinafter with reference to FIG. 10.

In FIG. 9 the load behaviors operation 906 receives the situationfeatures 905 created by feature extraction operation 902. It alsoreceives the strategy 907 extracted by the strategy extraction module.The strategy 907 is associated with situation features 905 by the loadbehaviors operation 906. The load operation 906 loads the associatedfeatures and strategy in behaviors information 812. This behaviorsinformation is used by the learning module 814 of FIG. 8, which is shownin detail in FIG. 11.

FIG. 10 illustrates the operational flow of the strategy extractionmodule 904 of FIG. 9. The operational flow for strategy extractionmodule begins with the retrieve operation 1002 and get operation 1004.Retrieve operation 1002 retrieves the situations and actions associatedwith situations from the observations information 804. Get operation1004 gets the method choices along with parameter limitations fromsubject knowledge information for the methods that may be used to workwith a given situation/actions combination.

The compare operation 1006 compares the situation/actions informationwith the method/parameters information and passes the results of thecomparison onto the select operation 1008. Based on the historicalassociation between the situation/actions and a method/parameterscombination, the selection operation 1008 selects a behavior methodalong with its parameters to be used with the situation/actionscombination by the user. The method and parameter values selected inoperation 1008 are then saved as one strategy iteration (method andparameter values) as strategy information 907 (FIG. 9) by save operation1010.

After the method/parameter selection is saved, actions ended test 1012detects whether there are more actions in the processing of thesituation that need to be evaluated. If there are more actions, then theoperation flow returns to retrieve operation 1002, which retrieves thenext situation/actions combination from the observations information.Compare operation 1006 then finds the best behavior method/parametercombination to be used with the situation/actions combination. Selectoperation 1008 then selects the best method and parameters for thesituation/actions combination and save operation 1010 saves that as thenext method/parameters iteration in the strategy information 907. Thiscyclic loop continues until all actions have been handled by thestrategy extraction operational flow. When all actions have beenprocessed, the operational flow returns to the load behaviors operation906 in FIG. 9, which then works with the situation features 905 andstrategy 907 extracted by the FIG. 10 operational flow.

In FIG. 11 the operational flow for the learning module 814 in FIG. 8 isshown. The operational flow in the learning module begins with constructoperation 1102 constructing a test deck. The test deck is constructedfrom expert behaviors in behaviors 812 created by behavior constructionoperation flow in FIG. 9. Selection of experts from a set of users ismainly a management decision process, probably using a consultingservice. The test deck 1104 contains two subsets of information. A firstsubset is a training subset. The training subset is used by create modelprofile operation 1106 to create a model profile 1108. Model profilecontains features/methods separation data,features/parameters-calculation-rules separation data, separationdecisions thresholds, decision applicability thresholds. The modelprofile 1108 is used by a verify model profile operation 1110 to verifythe effectiveness of the strategies in the model profile. This verifymodel operation uses a testing subset also created by constructoperation 1102 to test the performance of the model profile created fromthe training subset. This model performance is stored as model profileperformance 1112. Load profile operation 1114 then receives the modelprofile 1108 and the model profile performance 1112. It loads both intothe profile 310 for use by the behavior emulator module 308 in FIG. 3.This completes the description of the training cycle.

While the invention has been particularly shown and described withreference to preferred embodiments thereof, it will be understood bythose skilled in the art that various other changes in the form anddetails may be made therein without departing form the spirit and scopeof the invention.

1-10. (canceled)
 11. A computer method for emulating an expertdecision-making process based on an history of behaviors by humanexperts in a variety of observed allocation situations to provide a usera recommended allocation solution to an allocation situation to beprocessed, the method comprising: receiving situation datarepresentative of an allocation situation to be processed; extractingsituation features from the situation data; recognizing a behaviormethod from a pattern of situation features and identifying therecognized behavior method; calculating parameter values based on thesituation features for parameters in the recognized behavior method;executing the recognized behavior method on the situation data using theparameter values to provide a recommended allocation solution for theallocation situation; and displaying the recommended allocation solutionto the user.
 12. The method of claim 11 wherein the recommendedallocation solution comprises solution data representing a plan ofaction to provide the allocation solution and remainder datarepresenting unprocessed situation data.
 13. The method of claim 12further comprising: detecting whether the remainder data is in a targetrange; if the remainder data is not in the target range, recognizinganother behavior method from a pattern of situation features from theremainder data and identifying the another recognized behavior method;calculating parameter values, based on the situation features from theremainder data, for parameters in the another recognized behaviormethod; and executing the another recognized behavior method on theremainder data using the parameter values in the another recognizedbehavior method to recommend an allocation solution for the allocationsituation.
 14. The method of claim 13 wherein the acts of recognizing,calculating and executing for another recognized method are repeateduntil the act of detecting detects the remainder data is in the targetrange.
 15. The method of claim 11 wherein the displaying step furthercomprises: displaying a strategy, the strategy being all the recognizedbehavior methods with parameter values and the solution data andremainder data for each recognized method.
 16. The method of claim 11wherein the act of extracting features bases the extraction on featuresthat are significant in recognizing a behavior method.
 17. The method ofclaim 11 wherein the act of recognizing a method is based onfeature/method separation data in multidimensional space of featuresinto areas with each area associated with a method used by experts. 18.The method of claim 11 wherein the act of calculating parameter valuescomprises: recognizing a parameter calculation rule for each parameterin the recognized behavior method from a pattern of situation features;and calculating the parameter values using parameter calculation rulesusing situation features.
 19. The method of claim 18 wherein the act ofrecognizing a parameter calculation rule is based onfeature/parameter-calculation-rules separation data in multidimensionalspace of features into areas with each area associated with a parametercalculation rule used by experts.
 20. The method of claim 11 furthercomprising: capturing observations of actions taken by experts inanalyzing a plurality of allocation situations; constructing behaviorsfrom the observations, the behaviors indicating situation features andassociated methods with parameter values.
 21. The method of claim 20further comprising: learning a history of allocation situations andassociated behavior methods from the behaviors; and generating a profileof strategies from the history.
 22. The method of claim 15 furthercomprising: receiving adjustments to the strategy, the adjustments beingat least one of changes in recognized methods and changes in parametervalues.